Adaptive Tuboid Shapes for Action Recognition

Encoding local motion information using spatio-temporal features is a common approach in action recognition methods. These features are based on the information content inside subregions extracted at locations of interest in a video. In this paper, we propose a conceptually different approach to video feature extraction. We adopt an entropybased saliency framework and develop a method for estimating tube-like salient regions of flexible shape (i.e., tuboids). We suggest that the local shape of spatio-temporal subregions defined by changes in local information content can be used as a descriptor of the underlying motion. Our main goal in this paper is to introduce the concept of adaptive tuboid shapes as a local spatio-temporal descriptor. Our approach’s original idea is to use changes in local spatio-temporal information content to drive the tuboid’s shape deformation, and then use the tuboid’s shape as a local motion descriptor. Finally, we conduct a set of action recognition experiments on video sequences. Despite the relatively lower classification performance when compared to state-of-the-art action-recognition methods, our results indicate a good potential for the adaptive tuboid descriptor as an additional cue for action recognition algorithms.